Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve empty short_message identification #6

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Improve empty short_message identification #6

wants to merge 3 commits into from

Conversation

vasrem
Copy link

@vasrem vasrem commented Aug 12, 2019 •
edited
Loading

Handle cases where the short_message contains only control chars and spaces. This will lead to an error in the Graylog side.

Related issues

Reproduce

This can be reproduced by sending the following packet to a Graylog TCP or UDP input:

# TCP
echo -n '{ "version": "1.1", "host": "localhost", "short_message": "\n \n", "level": 5}' | nc -vw5 <IP> <PORT>
# UDP
echo -n '{ "version": "1.1", "host": "localhost", "short_message": "\n \n", "level": 5}' | nc -vuw5 <IP> <PORT>

Error output:

2019-08-12 09:04:05,522 ERROR   [DecodingProcessor] - Error processing message RawMessage{id=22de9723-bce0-11e9-a67f-f62dba2cc6c4, journalOffset=649416038, codec=gelf, payloadSize=1229, timestamp=2019-08-12T09:04:05.522Z, remoteAddress=/my_ip:26412} - {}
java.lang.IllegalArgumentException: GELF message <22de9723-bce0-11e9-a67f-f62dba2cc6c4> (received from <my_ip:26412>) has empty mandatory "short_message" field.
        at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:252) ~[graylog.jar:?]
        at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:134) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
        at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
        at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
        at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

Environment

Currently running this image on k8s 1.14

@vasrem vasrem force-pushed the short_message_ctrl_chars branch from d5cde1e to 5f350cf Compare August 12, 2019 10:16
@vwbusguy
Copy link

This would be excellent. I've found a few things that create this condition, including OpenShift logging and a Java appender for 3rd party code. If the message starts with any whitespace or new line, the short_message will end up empty, causing gelf to reject it, even if there is otherwise content. If it happens in a consistent way, we can intercept it in fluentd and add some string to the beginning of the message before it gets forwarded to gelf, but it's always a weird one to catch and debug.

@dstockman
Copy link

FYI - I think this project is no longer being maintained. I'd suggest someone fork and take it over.

invidian
invidian reviewed Oct 7, 2019
View reviewed changes
lib/fluent/gelf_util.rb Outdated Show resolved Hide resolved
lib/fluent/gelf_util.rb Outdated Show resolved Hide resolved
JTabel
JTabel reviewed Oct 16, 2019
View reviewed changes
lib/fluent/gelf_util.rb Outdated Show resolved Hide resolved
@vasrem
Copy link
Author

vasrem commented Oct 18, 2019 •
edited
Loading

@aayamauchi could you have a look to that? Seems like many people need it.

@toan-hf
Copy link

toan-hf commented Jun 25, 2020

@vasrem i think we can proactively fork this project also adding this feature and my feature here.

Looking forward to hearing from you.

Toan

@nix-power
Copy link

Any chance this change could be merged to master and tag 1.0.9 could be available for installation ?

@zolech
Copy link

zolech commented Jan 5, 2022

Please merge this!

@nix-power
Copy link

Can repo owner please merge this for version 1.0.9 ?
We really wait for this.

bmichalkiewicz added a commit to bmichalkiewicz/fluent-plugin-gelf-best that referenced this pull request Apr 17, 2024
@bmichalkiewicz
fix: Improve empty short_message identification hotschedules#6 by va…
f3f5319 
…srem
bmichalkiewicz added a commit to bmichalkiewicz/fluent-plugin-gelf-best that referenced this pull request Apr 17, 2024
@bmichalkiewicz
fix: Improve tests hotschedules#6 by vasrem
3792555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@invidian invidian invidian left review comments

@JTabel JTabel JTabel left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@vasrem @vwbusguy @dstockman @toan-hf @nix-power @zolech @JTabel @invidian